Journal Club

Jay Brophy MD PhD

Departments of Medicine, Epidemiology and Biostatistics, McGill University

2025-02-26

How to be a good critic of published article

  • be a critical reader
  • show your critique to be “tenable” 
  • remember ATOM (“Accept uncertainty. Be thoughtful, open, and modest.”)

What to avoid

  • Hit-and-run criticism: Pointing out flaws without providing alternative explanations. 
  • Dogmatic criticism: Relying on rigid principles (e.g., all unblinded RCTs are biased). 
  • Speculative criticism: Offering untested alternative explanations. 
  • Tubular criticism: Ignoring evidence that contradicts the critique.

Article # 1

Recent NEJM publication - Transcatheter Aortic-Valve Replacement for Asymptomatic Severe Aortic Stenosis    
 
METHODS At 75 centers in the United States and Canada, we randomly assigned, in a 1:1 ratio, patients with asymptomatic severe aortic stenosis to undergo early transcatheter aortic-valve replacement (TAVR) with transfemoral placement of a balloon-expandable valve or clinical surveillance. The primary end point was a composite of death, stroke, or unplanned hospitalization for cardiovascular causes. Superiority testing was performed in the intention-to-treat population. 

CONCLUSIONS Among patients with asymptomatic severe aortic stenosis, a strategy of early TAVR was superior to clinical surveillance in reducing the incidence of death, stroke, or unplanned hospitalization for cardiovascular causes. (Funded by Edwards Lifesciences; EARLY TAVR ClinicalTrials.gov number, NCT03042104

Results

 
Primary outcome (top left) composite of death, stroke, and unplanned cardiac hospitalization

Criticism

Trial criticized for being unblinded but evidence to support this criticism has been lacking

Let’s look closer at the results and do some simulations

Kaplan Meier Simulation

Approximating cumulative incidence to time to event data to perform Cox model analysis 

Simulated HR = 0.53, 95% CI 0.43, 0.66 reasonably close to published HR 0.50, 95% 0.40, 0.63  

The missing analysis and figure

Kaplan Meier Simulation Landmark Analysis

HR = 0.81 (95% CI 0.61, 1.07) p =0.14, IOW no benefit after 1 year

Is bias due to unblindness a tenable explanation?

Decision to seek medical care and the decision for hospitalization both have subjective components. There are 4 possible ways unbindedness could contribute to differences in hospitalizations.  

  1. patients in the clinical surveillance group knew they had severe disease, and knew they were not treated or “fixed.” This could increase their anxiety favor their conversion from asymptomatic to symptomatic patients—largely, because they knew they had severe disease and were unfixed. 

  2. medical staff also knew these patients were not “fixed” and thereby increasing theirprobability of attributing any symptoms, cardiac or not, to their underlying aortic stenosis 

  3. early surgery group knew they were “fixed” and would be less likely to consult for any symptoms cardiac or not 

  4. similarly medical staff of the early TAVR patients knowing they were “fixed” would be less likely to attribute any symptoms to heart disease 

Importantly, all 4 mechanisms would lead to an over-estimate of TAVR benefit in reducing hospitalizations

TAVR effect vs unblinding effect

Consider two Scenarios 

Scenario 1 benefit of decrease hospitalization occurs immediately following randomization, e.g. in the first year, with no longer term benefits 

Scenario 2 the benefit is present throughout the follow-up perhaps at a smaller initial rate but remains continuous, or perhaps even increases over time.  

Logic says Scenario 1 more likely for unblinding effect  

Logic says Scenario 2 is more likely for TAVR effect 

Landmark analysis compatible with Scenario 1 and therefore… 

Opinion also supported by the lack of benefit for the more objective outcomes of death or stroke. 

N.B. Further support comes from prior empirical evidence, see here and here, that lack of blinding exaggerates intervention effect estimates, especially in trials with subjective outcomes.

Article # 2

Recent NEJM publication - Colchicine in Acute Myocardial Infarction    
 
METHODS In this multicenter trial with a 2-by-2 factorial design, we randomly assigned patients who had myocardial infarction to receive either colchicine or placebo and either spironolactone or placebo. The results of the colchicine trial are reported here. The primary efficacy outcome was a composite of death from cardiovascular causes, recurrent myocardial infarction, stroke, or unplanned ischemia-driven coronary revascularization, evaluated in a time-to-event analysis. C-reactive protein was measured at 3 months in a subgroup of patients, and safety was also assessed. 

CONCLUSIONS Among patients who had myocardial infarction, treatment with colchicine, when started soon after myocardial infarction and continued for a median of 3 years, did not reduce the incidence of the composite primary outcome (death from cardiovascular causes, recurrent myocardial infarction, stroke, or unplanned ischemiadriven coronary revascularization). (Funded by the Canadian Institutes of Health Research and others; CLEAR ClinicalTrials.gov number, NCT03048825.)

Results

Interpretation

NEJM CONCLUSIONS Among patients who had myocardial infarction, treatment with colchicine, when started soon after myocardial infarction and continued for a median of 3 years, did not reduce the incidence of the composite primary outcome (death from cardiovascular causes, recurrent myocardial infarction, stroke, or unplanned ischemia driven coronary revascularization).  
 

PI to the press “I was a believer in colchicine, but after CLEAR I decided to stop it in my parent” 
 
 

Questions 
1. Why was he believer? (because of COLCOT?) 
2. This belief presumably wasn’t universally shared or the necessary equipoise would not have been present to proceed with the CLEAR trial

COLCOT - Another NEJM publication

Efficacy and Safety of Low-Dose Colchicine after Myocardial Infarction

NEJM CONCLUSIONS Among patients with a recent myocardial infarction, colchicine led to a significantly lower risk of ischemic cardiovascular events than placebo.

How to reconcile?

Do you just ignore the previous RCT of 4700 subjects?

How to reconcile?

Better might be to combine or borrow information between the studies 

CLEAR

COLCOT

How to reconcile?

COMBINED

How to be a good critic of published article

  • be a critical reader
  • show your critique to be “tenable” 
  • remembe